> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/mlfoundations/open_clip/llms.txt
> Use this file to discover all available pages before exploring further.

# Loading Models

> Learn how to load OpenCLIP models from various sources including pretrained weights, HuggingFace Hub, and local directories

OpenCLIP provides flexible model loading with support for pretrained weights, custom configurations, and multiple storage backends.

## Basic Model Loading

### create\_model()

The core function for creating CLIP models with flexible configuration options.

```python theme={null}
import open_clip

model = open_clip.create_model(
    'ViT-B-32',
    pretrained='laion2b_s34b_b79k',
    device='cuda',
    precision='fp16'
)
```

<ParamField path="model_name" type="str" required>
  Model architecture name (e.g., 'ViT-B-32', 'RN50') or schema-prefixed path:

  * Built-in: `'ViT-B-32'`
  * HuggingFace Hub: `'hf-hub:org/repo'`
  * Local directory: `'local-dir:/path/to/model'`
</ParamField>

<ParamField path="pretrained" type="str">
  Pretrained weights source. Can be:

  * Tag name (e.g., 'openai', 'laion2b\_s34b\_b79k')
  * Local file path (e.g., '/path/to/weights.pt')
  * Ignored if model\_name uses schema prefix
</ParamField>

<ParamField path="device" type="str | torch.device" default="cpu">
  Device to load model on ('cpu', 'cuda', etc.)
</ParamField>

<ParamField path="precision" type="str" default="fp32">
  Model precision: 'fp32', 'fp16', 'bf16', 'pure\_fp16', 'pure\_bf16'
</ParamField>

<ParamField path="jit" type="bool" default="False">
  Whether to JIT compile the model
</ParamField>

<ParamField path="force_image_size" type="int | Tuple[int, int]">
  Override default image size for the model
</ParamField>

<ParamField path="cache_dir" type="str">
  Directory for caching downloaded weights
</ParamField>

## Loading Schemas

### HuggingFace Hub

Load models directly from HuggingFace Hub using the `hf-hub:` schema:

```python theme={null}
model = open_clip.create_model(
    'hf-hub:laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K',
    device='cuda'
)
```

The function automatically:

* Downloads `open_clip_config.json` from the repo
* Looks for weights files (`.safetensors`, `.bin`, `.pth`)
* Merges preprocessing configuration

### Local Directory

Load from a local directory containing model config and weights:

```python theme={null}
model = open_clip.create_model(
    'local-dir:/path/to/my/model',
    device='cuda'
)
```

<Note>
  Local directory must contain:

  * `open_clip_config.json` with model configuration
  * Weight file (searched in order): `open_clip_model.safetensors`, `pytorch_model.bin`, `model.pth`, etc.
</Note>

### Local File Path

Load weights from a specific file:

```python theme={null}
model = open_clip.create_model(
    'ViT-B-32',
    pretrained='/path/to/checkpoint.pt',
    device='cuda'
)
```

## Advanced Loading Options

### Tower-Specific Weights

Load separate weights for image and text towers:

```python theme={null}
model = open_clip.create_model(
    'ViT-B-32',
    pretrained_image=True,  # Load default ImageNet weights
    pretrained_text=True,   # Load default LM weights
    pretrained_image_path='/path/to/vision.pt',  # Override with custom weights
    pretrained_text_path='/path/to/text.pt'
)
```

<ParamField path="pretrained_image" type="bool" default="False">
  Load default pretrained weights for image tower (timm models)
</ParamField>

<ParamField path="pretrained_text" type="bool" default="True">
  Load default pretrained weights for text tower (HuggingFace models)
</ParamField>

<ParamField path="pretrained_image_path" type="str">
  Path to custom image tower weights (loaded after full model)
</ParamField>

<ParamField path="pretrained_text_path" type="str">
  Path to custom text tower weights (loaded after full model)
</ParamField>

### Custom Model Configuration

Override model architecture parameters:

```python theme={null}
model = open_clip.create_model(
    'ViT-B-32',
    pretrained='laion2b_s34b_b79k',
    force_quick_gelu=True,
    force_patch_dropout=0.5,
    force_image_size=336,
    force_context_length=128
)
```

## create\_model\_and\_transforms()

Convenience function that returns model with preprocessing transforms:

```python theme={null}
model, preprocess_train, preprocess_val = open_clip.create_model_and_transforms(
    'ViT-B-32',
    pretrained='laion2b_s34b_b79k',
    device='cuda',
    precision='fp16'
)

# Use transforms
from PIL import Image
image = Image.open('example.jpg')
image_tensor = preprocess_val(image)
```

Returns a tuple of `(model, train_transform, val_transform)`. The transforms handle:

* Image resizing and cropping
* Normalization with correct mean/std
* Data augmentation (training only)

<Warning>
  Always use `model.eval()` before inference. Models are in training mode by default, which affects layers like BatchNorm.
</Warning>

## create\_model\_from\_pretrained()

Strictly requires pretrained weights (raises error if weights can't be loaded):

```python theme={null}
model, preprocess = open_clip.create_model_from_pretrained(
    'ViT-B-32',
    pretrained='laion2b_s34b_b79k',
    device='cuda',
    return_transform=True
)
```

<ParamField path="return_transform" type="bool" default="True">
  Whether to return preprocessing transform. If False, returns only model.
</ParamField>

This is the recommended function for inference use cases where pretrained weights are essential.

## Listing Available Models

```python theme={null}
import open_clip

# List all model architectures
architectures = open_clip.list_models()
print(architectures)  # ['RN50', 'RN101', 'ViT-B-32', 'ViT-L-14', ...]

# List all pretrained weights
pretrained = open_clip.list_pretrained()
for model_name, tag in pretrained:
    print(f"{model_name}:{tag}")

# List pretrained weights as strings
pretrained_str = open_clip.list_pretrained(as_str=True)
# ['RN50:openai', 'RN50:yfcc15m', 'ViT-B-32:laion2b_s34b_b79k', ...]
```

## Weight Loading Options

<ParamField path="load_weights" type="bool" default="True">
  Whether to load the resolved pretrained weights. Set to False for random initialization.
</ParamField>

<ParamField path="require_pretrained" type="bool" default="False">
  Raise error if pretrained weights cannot be loaded
</ParamField>

<ParamField path="weights_only" type="bool" default="True">
  Use `weights_only=True` for torch.load (safer, prevents arbitrary code execution)
</ParamField>

## Complete Example

```python theme={null}
import torch
import open_clip
from PIL import Image

# Load model with transforms
model, _, preprocess = open_clip.create_model_and_transforms(
    'ViT-L-14',
    pretrained='datacomp_xl_s13b_b90k',
    device='cuda',
    precision='fp16',
    force_image_size=224
)
model.eval()

# Get tokenizer
tokenizer = open_clip.get_tokenizer('ViT-L-14')

# Prepare inputs
image = preprocess(Image.open('cat.jpg')).unsqueeze(0).cuda()
text = tokenizer(["a cat", "a dog"]).cuda()

# Inference
with torch.no_grad(), torch.cuda.amp.autocast():
    image_features = model.encode_image(image)
    text_features = model.encode_text(text)
    
    # Normalize features
    image_features /= image_features.norm(dim=-1, keepdim=True)
    text_features /= text_features.norm(dim=-1, keepdim=True)
    
    # Compute similarity
    similarity = (100.0 * image_features @ text_features.T).softmax(dim=-1)
    print("Similarity:", similarity)
```